Ultracold atom system is proven to be an ideal platform for exploring exotic quantum state in many-body systems.^[1,2] Recently, the experimental realization of synthetic spin–orbit coupling (SOC) in both Bose–Einstein condensate (BEC) and quantum degenerate Fermi atomic gas provides us another window to explore novel states of matter, and has profoundly advanced our understanding of quantum phenomena in condensed-matter physics, such as topological insulators, quantum spin Hall effect, superconductivity, and so on.^[3–20] The main advantage of such a system lies in the fact that the SOC can be realized in a bosonic system, which is in stark contrast to the traditional condensed one, resulting in many novel quantum states that cannot be found in the traditional condensed-matter one.^[21–23]

Generally speaking, in real ultracold atom experiments, the atomic gas is trapped by an external potential. The ground state properties of a spin–orbit-coupled BEC trapped in a variety of external potentials, such as harmonic trap, optical lattice, concentrically coupled annular trap, and toroidal trap, have been well studied.^[7,24–26] With the development of technology, a new type of external potential, that is the harmonic plus quartic trap, can be realized by adding a quartic part to a harmonic potential. With the help of such potential, one can study the ground state and dynamical properties of BEC when the rotation frequency Ω exceeds the trapping frequency.^[27–31] Very recently, Chen and his coauthors have investigated the ground-state and rotational properties of BEC, with or without SOC, in such a type of external potential.^[32,33]

So far, most of previous works on spin–orbit coupled BEC have been restricted to SU(2) type SOC, where the internal states are coupled to their momenta via the SU(2) Pauli matrices and a variety of topological defects have been predicted.^[34–42] To completely describe all the pairwise couplings between different internal states in a spin-1 system, the SU(3) type SOC is more effective, where the spin operator is spanned by the Gell–Mann matrices.^[43–45] Very recently, the double-quantum spin vortices have been predicted in SU(3) spin–orbit-coupled Bose gases,^[46] in which the ground-state phases of the SU(3) spin–orbit coupled system are investigated for a homogeneous system.

Inspired by the works mentioned above, it is of particular interest to study the effects of both spin interaction and external potential on the ground state structure of such a system in different parameter regions, which is what we attempt to do in this work. The results show that different from the previously discussed SU(2) SOC, the ground-state structures of the trapped system depend strongly on such controllable system parameters.

The remainder of the present paper is organized as follows. In Section 2, we formulate the theoretical model describing a spin-1 BEC with SU(3) SOC confined in a harmonic plus quartic trap, and briefly introduce the numerical method. In Section 3, the ground-state structures of such a system for both ferromagnetic and antiferromagnetic spin interactions are presented, and the effects of SOC and the external potential are discussed. Finally, the main results of this paper are summarized in Section 4.

To begin with, we consider a quasi-two-dimensional (Q2D) spin-1 BEC with SU(3) SOC, which is confined in a harmonic plus quartic trap. In the zero-temperature mean-field theory, the energy functional of the system can be written as

In the present work, we assume that the spin-1 BEC with SU(3) SOC is confined by a harmonic plus quartic trap, which can be written as^[30]

The many-body ground states of the system can be obtained numerically by using the imaginary-time propagation method to solve the corresponding Gross–Pitaevskii equations, which can minimize the total energy functional.^[50,51] In our numerical calculations, we start from some proper initial wave-function, including the ground states of the homogeneous system, and propagate it until the fluctuation of the wave function becomes smaller than .

As is well-known, in the absence of SOC, the ground state of a spin-1 condensate is ferromagnetic for and antiferromagnetic for . In the presence of SU(3) SOC, the single-particle energy spectrum of the homogeneous system acquires three discrete minima residing on the vertices of an equilateral triangle, leading to a threefold-degenerate magnetized state or a topologically nontrivial lattice state, which strongly depends on the different choices among such three minima made by the many-body interactions.^[46] In what follows, we will investigate the ground state structure of a SU(3) spin–orbit-coupled BEC confined in a harmonic plus quartic trap for ferromagnetic and antiferromagnetic spin interactions, respectively. For simplicity and without loss of generality, we will consider the special case with and μ=0.1 throughout this work.^[28,30,32]

3.1. Ferromagnetic spin interaction

We first consider the ferromagnetic spin interaction, i.e., . In this case, the ground state of the spin-1 ferromagnetic condensate with SU(2) SOC is a plane wave state with a finite momentum.^[7] Figure 1 shows the ground-state density, phase, spin, and momentum distributions of ferromagnetic interaction with for weak SU(3) SOC, where the momentum distribution can be given by using the Fourier transformation . As shown in Fig. 1(b), the phase distribution of each component and momentum distribution demonstrate that the ground state of a trapped SU(3) system is also a single plane wave state, where only one minimum of the single-particle spectrum is occupied through spontaneous symmetry breaking. This is consistent with the homogenous SU(3) system in which one of the three minima of the single-particle spectrum is selected by spontaneous symmetry breaking, and the system shows threefold degenerate instead of doubly degenerate in the SU(2) case.

Fig. 1.

Figure Option ViewDownloadNew Window

Fig. 1. The ground state of a spin-1 ferromagnetic BEC with SU(3) SOC for , , and κ=1. (a) The density distribution of each component and the total density distribution of the system. (b) The phase distribution of each component and momentum space distribution. (c) The spin-component distribution and spin texture of the system. The strength of the quartic trap μ=0.1, and the field of view is 6.4 ×6.4 in dimensionless units with .

However, if we look at the density distributions of the system shown in Fig. 1(a), we find that the system is phase separation. It is in stark contrast to the homogenous case, where three components are phase coexistence or miscible, and the system is in a magnetized phase with the spatial transitional symmetry preserved but the time-reversal symmetry broken.^[46] To give a more clear explanation of the above results, we also plot the spin-component distribution and spin texture of the system,^[47,48] which can be written as

Figure 1(c) exhibits the spin-component distribution and the spin texture of the system. From the z-component distribution of spin, one can easily find the phase separation behavior of the trapped SU(3) system.

For a relatively strong SU(3) SOC, such as κ=4.5 shown in Fig. 2, one find a similar, but more pronounced ground-state density, phase, spin, and momentum space distributions. In this case, it is easy to see that the momentum of the plane wave increases with the strength of the SU(3) SOC, as shown in Fig. 2(b) for the phase distribution of each component.

Fig. 2.

Figure Option ViewDownloadNew Window

Fig. 2. The ground state of a spin-1 ferromagnetic BEC with SU(3) SOC for , , and κ=4.5. (a) The density distribution of each component and the total density distribution of the system. (b) The phase distribution of each component and momentum space distribution. (c) The spin-component distribution and spin texture of the system. The strength of the quartic trap μ=0.1, and the field of view is 6.4 ×6.4 in dimensionless units with .

Figure 3 shows another threefold-degenerate ground state of the system for strong SU(3) SOC κ=6. In this case, the ground state of the system is also a plane wave, and only one of the three minima of the single-particle spectrum is occupied. However, the spin component distribution is totally different from that in the previous cases, where the x- and y-components of the spin distribution show a funnel structure, as shown in Fig. 3(c). For the last choice of the threefold-degenerate ground state, that is the lower right corner of the three minima of the single-particle spectrum is occupied, the density and spin distributions are similar to those in Fig. 3.

Fig. 3.

Figure Option ViewDownloadNew Window

Fig. 3. The ground state of a spin-1 ferromagnetic BEC with SU(3) SOC for , , and κ=6. (a) The density distribution of each component and the total density distribution of the system. (b) The phase distribution of each component and momentum space distribution. (c) The spin-component distribution and spin texture of the system. The strength of the quartic trap μ=0.1, and the field of view is 6.4 ×6.4 in dimensionless units with .

3.2. Antiferromagnetic spin interaction

We now turn our attention to the case of antiferromagnetic spin interaction, that is, . We recall that for the SU(2) spin–orbit-coupled system, only two points in momentum space of the single-particle spectrum are selected, leading to the formation of stripe phase.^[7] Figure 4 shows the ground-state density, phase, spin, and momentum space distributions of a spin-1 antiferromagnetic condensate for and with weak SU(3) SOC κ=1. As shown in Fig. 4(a), the density of each component exhibits obviously phase separation, where three discrete minima with unequal weights form a equilateral triangle in momentum space (as shown in the last column of Fig. 4(b)). This is in stark contrast to the homogeneous case, where three discrete minima with equal weights are selected.

Fig. 4.

Figure Option ViewDownloadNew Window

Fig. 4. The ground state of a spin-1 antiferromagnetic BEC with SU(3) SOC for , , and κ=1. (a) The density distribution of each component and the total density distribution of the system. (b) The phase distribution of each component and momentum space distribution. (c) The spin-component distribution and spin texture of the system. The strength of the quartic trap μ=0.1, and the field of view is 6.4 ×6.4 in dimensionless units with .

For strong SU(3) SOC, such as κ=4.5 shown in Fig. 5, the density of each component displays a hexagonal honeycomb lattice structure, as shown in Fig. 5(a). If we look at the momentum space distribution shown in the last column of Fig. 5(b), three discrete minima with equal weights are occupied due to the presence of the strong SU(3) SOC, which is consistent with the homogeneous case.

Fig. 5.

Figure Option ViewDownloadNew Window

Fig. 5. The ground state of a spin-1 antiferromagnetic BEC with SU(3) SOC for , , and κ=4.5. (a) The density distribution of each component and the total density distribution of the system. (b) The phase distribution of each component and momentum space distribution. (c) The spin-component distribution and spin texture of the system. The strength of the quartic trap μ=0.1, and the field of view is 6.4 ×6.4 in dimensionless units with .

Finally, we give an experimental protocol to observe the above ground-state phases in future experiments. The most important parameter of the system considered in this work is the SU(3) SOC. Following the Raman dressing method used in NIST experiments,^[5,52] the SU(3) SOC can be experimentally realized by using a similar method of Raman dressing, where three laser beams with different polarizations and frequencies intersect at an angle of 2 π/3. In addition, the strength of the SU(3) SOC can also be precisely controlled by optical means. With regard to the contact spin interactions, the ferromagnetic and antiferromagnetic spin interactions can be realized in ⁸⁷Rb and ²³Na condensates, respectively. For the quasi-two dimensional system, the dimensionless spin interaction parameters are usually written as and ,^[53] where N is the particle number, is the characteristic length of the harmonic potential along z direction with being the trapping frequency and serves as the unit of spatial coordinate. If we take the real experimental parameters and , then and correspond to and , which are actually so weak that the mean field theory used in this paper is still valid. Futhermore, when the strength of the quartic trap is relatively weak, such as μ=0.01 or even smaller, the ground-state of the system shows qualitatively similar structures with the above results, leaving the main physics analyzed in this work unaltered.

In summary, we have considered a spin-1 BEC with SU(3) SOC confined in a harmonic plus quartic trap. The effects of SU(3) SOC on the ground-state structure of the system are investigated for both ferromagnetic and antiferromagnetic condensates. Different from the system with SU(2) Rashba SOC, where the single-particle ground states are infinitely degenerate along a circular ring in momentum space, the SU(3) SOC leads to a threefold-degenerate single particle ground state in momentum space. When the atomic interactions are taken into account, the SU(3) SOC gives rise to some novel quantum states, which are different from the plane wave and stripe phase generated by the SU(2) SOC. For the ferromagnetic spin interaction in the condensate with SU(3) SOC, the ground state shows a threefold-degenerate plane wave ground state, but with a different spin-component distribution for strong SOC. While for the antiferromagnetic spin interaction, three discrete minima with unequal (equal) weights in momentum space are selected for weak (strong) SU(3) SOC, and a hexagonal honeycomb lattice structure is formed for strong SU(3) SOC. These results show that the ground-state structure of a trapped SU(3) spin–orbit coupled BEC has strong dependence on both the strength of the SU(3) SOC and the spin-exchange interaction.